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^D . Abstract 

(N ■ 

Logit Dynamics [Blume, Games and Economic Behavior, 1993] is a randomized best response 

dynamics for strategic games: at every time step a player is selected uniformly at random and she 

chooses a new strategy according to a probability distribution biased toward strategies promising 

fvj ' higher payoffs. This process defines an ergodic Markov chain, over the set of strategy profiles of 

fv^ ' the game, whose unique stationary distribution is the long-term equilibrium concept for the game. 

However, when the mixing time of the chain is large (e.g. exponential in the number of players) , the 

stationary distribution loses its appeal as equilibrium concept, and the transient phase of the Markov 

chain becomes important. In several cases it happens that on a time-scale shorter than mixing time 

[^ , the chain is "quasi-stationary", meaning that it stays close to some small set of the state space, 

while in a time-scale multiple of the mixing time it jumps from one quasi-stationary configuration 

Q_j I to another; this phenomenon is usually called "metastability". 

In this paper we give a quantitative definition of "metastable probabifity distributions" for a 
Markov chain and we study the metastability of the Logit dynamics for some classes of coordination 
games. In particular, we study no-risk-dominant coordination games on the cfique (which is equiv- 
alent to the well-known Glauber dynamics for the Ising model) and coordination games on a ring 
*K^ ' (both the risk-dominant and no-risk-dominant case). We also describe a simple "artificial" game 

l^-s , that highlights the distinctive features of our metastability notion based on distributions. 
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1 Introduction 

Complex systems consist of a large number of components that, at a small scale interact according to 
simple rules and, despite of this, exhibit complex large scale behaviors. Complex systems can be found 
in Economics (e.g., the market). Physics (e.g., ideal gases, spin systems). Biology (e.g. evolution of life) 
and Computer Science (e.g., Internet and social networks). Analyzing, understanding how such systems 
evolve, and predicting their future states is a major research endeavor. 

In this paper we focus on selfish systems in which the components (called the players) are selfish 
agents, each one with a set of possible actions or strategies trying to maximize her own payoff. The 
payoff obtained by each player depends not only on her decision but also on the decisions of the other 
players. We study a specific dynamics, the logit dynamics [6], and consider as solution concept its 
equilibrium states. Logit dynamics is a type of noisy best response dynamics that models in a clean and 
tractable way the limited knowledge (or bounded rationality) of the players in terms of a parameter /3 
(in similar models studied in Physics, /3 is the inverse of the temperature). Intuitively, a low value of 
/3 (that is, high temperature and entropy) represents the situation where players choose their strategies 
"nearly at random" ; a high value of /3 (that is, low temperature and entropy) represents the situation 
where players "almost surely" play their best response; that is, they pick the strategies yielding high 
payoff with higher probability. It is well known that this dynamics induces, for every strategic game, 
a Markov chain which has a unique stationary distribution (the Markov chain is ergodic). Thus no 
equilibrium selection problem arises. The drawback of using the stationary distribution to describe the 
system behaviour is that the system may take too long to reach it, unless the chain is rapidly mixing. 
Logit dynamics for strategic games can be rapidly mixing or not^ depending on the features of the 
underlying game, the temperature/noise and the number of players [21I2]- For this reason, in this work 
we focus on the transient phase of the logit dynamics and, in particular, we try to answer the following 
questions: when the mixing time is exponential, the transient phase is completely chaotic, or we can 
still spot some regularities? Are we able to say something about the behavior of the chain before it 
reaches the stationary distribution? Obviously, such a chain is perfectly described by the collections 
of probability distributions consisting of one distribution for each time step and each starting profile. 
This should be contrasted with the rapidly mixing case (i.e., a Markov chain with polynomial mixing 
time) in which one can approximately describe the state of the system (after the mixing time) using one 
distribution (that is, the stationary distribution). 

Our results show games for which regularities can be observed even in the transient phase. In par- 
ticular, we will show that, depending on the starting profile, the dynamics rapidly reaches a distribution 
and remains close to this distribution for a sufficiently long time (we call such a distribution metastable) . 
We can describe our results also in terms of the quantity of information needed to predict the status of 
a system that evolves in time according to the logit dynamics. We know that the long-term behavior 
of the system can be compactly described in terms of a unique distribution but we have to wait a tran- 
sient phase of length equal to the mixing time. Thus, if the system is rapidly mixing this description 
is significant after a short transient phase. However, when the mixing time is super-polynomial this 
description becomes significant only after a long time. Our results show that for a large class of n-player 
games logit dynamics is not rapidly mixing but the profile (the strategies played by the n-players) can 
still be described with good approximation and for a super-polynomial number of steps by means of 
a small number of probability distributions. This comes at the price of sacrificing a short polynomial 
initial transient phase (so far we are on a par with the rapidly mixing case) and requires a few bits of 
information about the starting profile (this is not needed in the rapidly mixing case). 

Our Results. In this work, we obtain results about the metastability of the logit dynamics for different 
classes of coordination games. 

- We start in Section[3]by introducing the notion of an (e, T)-metastable distribution /i and of its pseudo- 
mixing time. Roughly speaking, /i is (e, T)-metastable for a Markov chain if, starting from /i, the Markov 
chain stays at distance at most e from fi for at least T steps. The pseudo-mixing time of /i starting from 
a state x, t^{e), is the number of steps needed by the Markov chain to get e-close to /i when started from 

X. 

In a rapidly-mixing Markov chain, after a "short time" and regardless of the starting state the chain 
converges rapidly to the stationary distribution and remains there. For the case of non-rapidly-mixing 



Markov chains, we replace the notions of "mixing time" and "stationary distribution" by that of "pseudo- 
mixing time" and that of "metastable distribution" . Intuitively speaking, we would like to say that, even 
when the mixing time is (prohibitively) high, there are "few" distributions which give us an accurate 
description of the chain over a "reasonable amount of time" . Roughly speaking, the state space i7 can be 
partitioned into a small number of subsets fii, 02, • ■ • of "equivalent" states; that is, if the chain starts in 
any of the states in 17^, then it will rapidly converge to a "metastable" distribution ^i, where metastable 
denotes the fact that the chain remains there for "sufficiently" long. 

- In Section |H we analyze the metastable distributions of the Ising model on the complete graph, also 
known as the Curie- Weiss model, and used by physicists to model the interaction between magnets in a 
ferro-magnetic system [26] . The model describes also the process of opinion formation and the spread 
of new technologies in fully connected societies (see, for example [IH])- It is known [T51I271I2] that this 
model can be seen as a game played by magnets and, in particular, the Glauber dynamics for the Gibbs 
measure on the Ising model is equivalent to the logit dynamics for this game where /3 is exactly the 
inverse of the temperature. The mixing time of this dynamics is known to be exponential for every 
(3 > 1/n. For this model, we show that distributions where all magnets have the same magnetization are 
(1/n, i)-metastable for any t = poly{n) when /3 = n{\ogn/n). Moreover we show that the pseudo-mixing 
time of these distributions is polynomial when the dynamics starts from a profile where the difference in 
the number of positive and negative magnets is large. 

- Graphical coordination games are often used to model the spread of a new technology in a social 
network [301 127] with the strategy of maximum potential corresponding to adopting the new technology; 
players prefer to choose the same technology as their neighbors and the new technology is at least as 
preferable as the old one. In Section [5] we follow [TS] and study the case in which social interaction 
between the players is described by the ring topology. We show that for every starting profile there is a 
metastable distribution and the dynamics approaches it in a polynomial number of steps. 

Finally, we consider the OR-game, an artificial game defined in [3], that highlights the distinctive features 
of our metastability notion based on distributions. 

Other equilibria notions. Let us compare the solution concept studied in this paper (the stationary 
distribution of the logit dynamics) with the notion of Nash equilibrium. Similar comparison can be made 
with other solution concepts from Game Theory (correlated equilibria, sink equilibria) . The notion of a 
Nash equilibrium has been extensively studied as a solution concept for predicting the behavior of selfish 
players. Indeed, if the players happen to be in a (Pure) Nash equilibrium any sequence of selfish best 
response (i.e., utility improving) moves keeps the players in the same state. Unfortunately, the theory of 
Nash equilibria does not explain how a Nash equilibrium is reached if players do not start from one and, 
in case multiple equilibria exist, does not say which equilibrium is selected (about this important issue 
see [13). Even tough it is not hard to see that sequences of best response moves may reach a Pure Nash 
equilibrium (if it exists) , recent hardness results regarding the computation of Nash equilibria |12l lll| 
suggest that Best Response, or any other dynamics, might take super-polynomial time in the number of 
players to reach an equilibrium. Thus, even in case only one (Pure) Nash equilibrium exists, the players 
might take very long to reach it and thus it cannot be taken to describe the state of the players (unless 
we are willing to ignore the super-polynomially long transient phase). In contrast, in the setting studied 
in this paper these drawbacks disappear: the solution concept is defined in terms of a dynamics and 
for each dynamics we have a unique equilibrium. For rapidly mixing chains the equilibrium is quickly 
reached. The results in this paper show that, even for the non-rapidly mixing case, the system can be 
described if we discount the initial transient phase. 

The Nash Equilibrium concept is based on the assumption that each player has complete information 
about the game and the strategies of his opponents and it is able to compute his best strategy with respect 
to the strategies played by the other players. However, in many complex systems, environmental factors 
can influence the way each agent selects her own strategy and limitations to the players' computational 
power can influence their behaviors. Logit dynamics is a clear and crispy way to model these settings. 

Related work. Logit dynamics has been first studied by Blume [6j who showed that, for 2x2 coor- 
dination games, the long-term behavior of the system is concentrated in the risk dominant equilibrium 
(see [in])- Ellison [TS] studied logit dynamics for graphical coordination games on rings (the class of 
games we study in Section [5]) and showed that some large fraction of the players will eventually choose 



the strategy with maximum potential. Similar results were obtained by Peyton Young [5D] for more 
general families of graphs. Montanari and Saberi [571 gave bounds on the hitting time of the highest 
potential equilibrium for the logit dynamics in terms of some graph theoretic properties of the underlying 
interaction network. Asadpour and Saberi 1 studied the hitting time of the Nash equilibrium for a class 
of congestion games. 

The study of the mixing time of logit dynamics for strategic games has been initiated in [3] (see 
also [5]), whose results highlight a separation between games where the mixing time can be bounded 
independently from the parameter (3 and games where the mixing time is necessarily exponential in /?. 
The Ising model is a very well-studied topic and we refer the reader to the survey of Martinelli [26 and 
to Chapter 15 of [25]. 

In Physics, Chemistry or Biology, metastability is a phenomenon related to the evolution of systems 
under noisy dynamics. In particular, metastability concerns about moves between regions of the state 
spaces and the existence of multiple, well separated time scales: at short time scales the system appears 
to be in a quasi- equilibrium, but really explores only a confined region of the available space state, while, 
at larger time scales, it undergoes transitions between such different regions. Examples of metastability 
can be found in biology, climatology, economics, materials science and physics. 

Metastability appear for the first time around 1935 with the work of Eyring [TO and Kramers jH] on 
diffusion in potential wells, but the mathematically rigorous analysis of metastability phenomena in the 
context of randomly perturbed dynamical systems start in the early 1970's with the work of Freidlin and 
Wentzell [TJ . Since then, metastability is a very well studied topic in Physics and several monographs on 
this subject are available (see, for example |20l [28l [Tj [21] ) . The goal of metastability is to model processes 
showing the following typical behavior: starting from a given profile, the system will rather quickly visit 
the nearby maximum of the potential function (a metastable state); the dynamics stays very close to 
such a state for a very long time, avoiding visits to other local maxima; at some point, the system leaves 
the metastable state (and its neighborhood) and moves to some other local maximum, usually better 
than the previous one; the process then is repeated. Research in Physics about metastability aims at 
expressing typical features of a metastable state and to evaluate the transition time between metastable 
states; the main approaches used to this analysis are based on large deviation theory [T^ or on potential 
theory [5] . Our approach is closest to the one of Bovier et al. [H] . They define the notion of a metastable 
■point as a state that is quickly reached and difficult to leave. For every metastable point x they define 
the local valley of x as the set of states for which x is the metastable point with the smallest hitting time 
and the associated metastable distribution associated with x is the stationary distribution restricted to 
the local valley. In [TU] , Bovier and Manzo apply the approach of [5] in the context of zero temperature 
limit of Glauber dynamics of spin systems in finite volume and show that the transition times can be 
expressed in terms of properties of the potential function. 

Metastability was analysed not only for discrete dynamics, but also for continuous Markov processes. 
In [23] Larralde et al. define a metastable state by two components: spectral feature of a state (namely, 
isolated eigenstate of the master operator of the Markov Process having an exceptionally low eigenvalue) 
and the technical condition meaning that the probability of being in a metastable state at equilibrium is 
vanishingly small. These conditions partition the state space in two disjoint set: the metastable states 
and the equilibrium states. They show that for any starting profile x, the dynamics quickly reach, 
with a probability px, a state which is fully in the metastable region and, with probability 1 — Px, the 
equilibrium. Further, if the dynamics start from the metastable region, then the probability of leaving it 
in short time is very low. Moreover, they consider a restricted dynamics in which the process is reflected 
each time it attempts to leave the metastable region, whose equilibrium is described by the restriction 
of the stationary distribution to the metastable region: they show that these restricted dynamics well 
mimic the process when the starting point is in the metastable region. 

Very recently, Beltran and Landim [3] describe for the continuous time Markov process of the Ising 
model all metastable behaviors, defining time scales at which they occur, the metastable set associated 
to each time scale, and the asymptotic dynamics which specifies at which rate the process jumps from 
one metastable state to another. 

The work on censored Glauber dynamics [241 [T3l [T4] is also related to ours: the mixing time in a 
censored dynamics resemble the pseudo mixing time for the metastable distribution on a subset of states. 
However, we stress that the censored dynamics alters the original evolution of the Markov chain and the 
techniques developed do not seem useful to answer questions about the pseudo-mixing time. 



Notations. We write \S\ for the size of a set S. We use bold symbols for vectors; when x = 
(xi, . . . ,Xn) £ {0, 1}" we write |x|a for the number of occurrence of a in x; i.e., |x|a = [{« G [n] : Xi = a}\. 
We use the standard game theoretic notation (x_i,y) to mean the vector obtained from x by replacing 
the i-th entry with y; i.e., (x_i, y) — (xi, . . . , Xi-i, y, Xi+i, . . . , Xn)- We denote by negl {n) a function in 
n that is smaller than the inverse of every polynomial in n. For distributions /i and v on the same space 
i7, the total variation distance between (i and v is defined as l|/z — i^||rpy :— ^ 'Ylx^o, \l^i^) ~ ^^(2^)1- 

2 Logit dynamics 

A strategic game 5 is a triple Q = {[n\,S^lA), where [n] = {l,...,n} is a finite set of players, S = 
{Si, . . . , Sn} is a family of non-empty finite sets {Si is the set of strategies for player i), and U — 
{mi, . . . , u„} is a family of utility Junctions (or payoffs), where Ui : 5*1 x • • ■ x S*,! ^^ R is the utility 
function of player i. 

Logit dynamics is a randomized best-response dynamics where at every time step a players i G [n] 
is selected uniformly at random, and she updates her strategy according to the following probability 
distribution over the set Si. For every y G Si 

a,{y\^)^-±-e^-^(-~"y^ (1) 

Ti{x) 

where x G 5i x • ■ • x 5„ is the strategy profile played at the current time step, Ti(x) 
~ Szes g/3ui(x_i,z) jg ^Yie normalizing factor, and /3 ^ 0. 

Parameter /3 is a rationality level for the system, indeed from ([Ij it is easy to see that, for /3 = (no 
rationality) player i selects her strategy uniformly at random, for /3 > the probability is biased toward 
strategies promising higher payoffs, and for /3 — > oo (full rationality) player i chooses her best response 
strategy (if more than one best response is available, she chooses uniformly at random one of them). 

For every /3 ^ 0, the above dynamics defines a Markov chain with the set of strategy profiles as state 
space, where the transition probability from profile x — (xi, . . . , x„) to profile y — (j/i, . . . , 2/„) is zero if 
the X and y differ at more than one player, and it is —<Ji{yi \ x) if the two profiles differ exactly at player 
i. More formally, we can define the logit dynamics as follows. 

Definition 2.1 (Logit dynamics ^) Let Q — {[n\,S,U) he a strategic game and let /3 ^ 0. The logit 
dynamics for Q is the Markov chain Mp = {Xt : t G N} with state space il = 5*1 x ■ • • x 5„ and transition 
matrix 

<7i{Vi I x), ify-i = x-i and yi^Xi\ 

^(x,y) = --<! Er=i^.(2/dx), */y = x; (2) 



n 



0, otherwise; 



where ai{yi \ x) is defined in (Qp. 



It is easy to see that the Markov chain is ergodic, indeed P"(x, y) > for every pair of profiles x and 
y. Hence, a unique stationary distribution tt exists and for every starting state x the distribution of the 
chain P*(x, •) at time t approaches tt as i tends to infinity. 

Potential Games. A game Q = ([n],5,iY) is said a (exact) potential game if a function $ : 5*1 x 
■ ■ ■ X Sn — !• M exists such that, for every player i and for every pair of profiles x and y that differ 
only at position i, it holds that Ui(x) — Wi(y) = *&(x) — '^(y)- It is easy to see that, ii G ^ {[n],S,'U) 
is a potential game with potential function $, then the Markov chain given by ([2]) is reversible (i.e. 
7r(x)P(x, y) = 7r(y)P(y,x) for every x and y) and its stationary distribution is the Gibbs measure 

^(x) = le^*^'') (3) 

where Z = X^ves x-xs e*^**^^' is the normalizing constant. 

When ^ is a potential game, the logit dynamics is equivalent to the well-studied Glauber dynamics, 

meaning that logit dynamics for Q and Glauber dynamics for the Gibbs distribution tt in ^ define 



the same Markov chain. Due to the analogies between logit and Glauber dynamics, we will sometimes 
adopt the terminology used by physicists to indicate the quantities involved; in particular we will call 
parameter /? the inverse noise or inverse temperature and we will call partition Junction the normalizing 
constant Z of the Gibbs distribution ^. All games we consider in this paper are potential games. 



3 Metastability 

In this section we give formal definitions of metastable distributions and pseudo-mixing. As a simple 
example we analyze metastability for the logit dynamics for 2-player coordination games and we also 
highlight some connections between metastability and the bottleneck ratio. 

Definition 3.1 (Metastable distribution) Let P be a Markov chain with finite state space fl. A 
probability distribution ii over f2 is (e, r)-metastable if for every ^ t ^ T it holds that 



UiP'-^i 



TV 



^£ 



Here are two obvious property of metastable distributions. 

1. Monotonicity: If fi is (e, T)-metastable then it is (£',T')-metastable for every e' ^ e and T' ^ T; 

2. Stationarity and Metastability: if fi is (0, l)-metastable, then it is (0, T)-metastable for every T; fi 
is stationary if and only if it is (0, l)-metastable 

A third property is given by the following easy and useful lemma. 

Lemma 3.2 ///i is (e, l)-metastable for P then fj, is {eT,T) -metastable for P. 

Proof. By using the triangle inequality, we have 

yp^ - mILv ^ IIm^* - m^IItv + iIa^^ - a^iItv ^ IIm^*"' 



M 



■£, 



IITV ^ 11^^ ^ IITV "^ f-IITV ^ llf*" f^llTV 

where the last inequality follows from the (e, l)-metastability of fi and from the fact that if fi and ly are 
two probability distributions and P is a stochastic matrix then \\fiP — iyP\\r^y ^ ||/i — j/||rpy. D 

The definition of metastable distribution captures the idea of a distribution that behaves approxi- 
mately like the stationary distribution, meaning that if we start from such distribution and run the chain 
we stay close to it for a long time. 

Among all metastable distributions, we are interested in the ones that are quickly reached from a, 
possibly large, set of states. This motivates the following definition. 

Definition 3.3 (Pseudo-mixing time) Let P be a Markov chain with state space fi, let S C ^ be a 

set of states and let fi be a probability distribution over il. We define tf (e) as 

f^(£) = inf{t e N : ||P*(a;, •) - fi\\^^ ^ e for all x e S} 

We observe that the stationary distribution n of an ergodic Markov chain is reached within e in time 
tmixi^) from every state. Thus, according to Definition 13.31 we have that t^{e) ~ imix(£)- 
The following simple lemma connects metastability and pseudo-mixing. 

Lemma 3.4 Let fi be a {e,T) -metastable distribution and /et S* C 51 be a set of states such that tf^{e) < 
+0O. Then for every x ^ S it holds that 

||P*(x, •) - a^IItv < 2e for every tl{e) ^ t s^ tl{e) + T 

Proof. Let us name t ~ t — t^{e) for convenience sake. By using triangle inequality for the total variation 
distance, the fact that P* is a stochastic matrix, and the definitions of metastable distribution and 
pseudo-mixing, we have that 



|P*(x,-)-/i| 



TV 



< 



< 



p*m(^)(x,-)p*- 


/^ 


TV 










P*j:(^)(^,.)P*„^P* 


+ 
TV 


fiP 


*-A^ 


TV 


P*^(^)(a;,-)-M 


TV 


^ + 


MP* 


-A* 


TV 
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3.1 Example: A simple three-state Markov chain 





As a first example, let us consider the simplest Markov chain that may highlight the concepts of metasta- 
bility and pseudo- mixing, 

e 



p 



l-e 1-e 

The chain is ergodic with stationary distribution tt — (e, (1 — e)/2, (1 — £)/2), and its mixing time is 
^mix = (l/^)- Hence the mixing time increases as e tends to zero. 

Now observe that, for every S > e, degenerat(U distributions fj,i = (0,1,0) and ^2 — (0,0,1) are 
{6, 8((5/e))-metastable according to Definition 13. II And if we start from the first state (i.e. from degen- 
erate distribution v — (1,0,0)), after one step we are in the stationary distribution. 

Hence, even if the mixing time can be arbitrary large, for every starting state x there is a (1/4, 0(irnix))- 
metastable distribution /i that is quickly (in constant time, independent of e) reached from x. 

3.2 Example: Two-player coordination games 

Coordination games with two players are examples of games where the mixing time is a function increasing 
exponentially in f3. The transition matrix of the logit dynamics for such games is 



P 



where 



/ 


(0,0) 


(0,1) 


(1,0) 


(1,1) \ 


(0,0) 


l-e 


e/2 


e/2 





(0,1) 


(l-e)/2 


ie + S)/2 





{l-S)/2 


(1,0) 


(l-e)/2 





{s + S)/2 


{l-S)/2 


V (1,1) 





S/2 


S/2 


l-<5 J 



1 



1 



I _|_ g(a-d)0 



1 + e(''-=)/3 



and a, b, c, d are the parameters of the coordination game with a > d, b > c, and a — d > b — c (see 
Section 4.1 in [3^ for further details). The stationary distribution for P is 

[5{l~e),e6,e6,e{l-d)] 



e + S 



In [3] we proved that the mixing time for such games is imix = ©(l/i^)- 

Now consider the special case when e — 6, hence the stationary distribution is 

^ = ((l-e)/2, e/2, e/2, (l-£)/2) 

Let /i(o,o) ^-i^d /^(i,i) be the two distributions concentrated in states (0,0) and (1, 1) respectively, i.e. 

M(o,o) = [1,0,0,0] and Ai(i,i) = [0,0,0,1] 

Observe that, if we start from /i(o,o) or /i(i.i), after one step of the chain we are respectively in distributions 

/i(o,o)^ = [1 - £, e/2, e/2, 0] and /i(i,i)^ = [0, e/2, e/2, 1 - e] 

Hence 

||a^(o,o)-P - M(o,o) IItv ^ 11^(1,1)^ ~ ^(1.1) IItv ^ ^ 



^ A probability distribution is degenerate if it is concentrated in one single element 



By using Lemma [331 we have that ^(o,o) and M(i,i) ^^'^ (1/4, 9(l/e))-metastable accordmg to Defini- 
tion [3ITJ Moreover, if the chain starts from state (0, 1) or from state (1,0), after 1 step of the chain we 
are e-close to the stationary distribution tt, indeed 



(0,1,0,0)P = 
and 



1-e 1-e 



(0,0,1,0)F = 



1-e l-e 

—-—, 0, e, ^-— 



||(0,l,0,0)P-7r||Tv = ll(0,l,0,0)P-^||Tv=e- 
We can summarize what we have just shown in the foUowing theorem. 

Theorem 3.5 Let P be the transition matrix of the logit dynamics for a 2-player coordination game. For 
every starting profile x G fJ there is a (1/4, Q{tmix))-'metastable distribution /ix such that tj^ =^ Q(l)- 

3.3 Metastability and the bottleneck ratio 

Consider an ergodic Markov chain P with state space Q and stationary distribution tt. For a subset S 
of states, the bottleneck ratio B{S) is defined as 

B{S) = ^^ where Q{S, S)^Y. J2 ^i^)Pi^^ v) ■ 

Let TTs be the stationary distribution conditioned on 5, i.e. 

^ .^N N(a;)/7r(5), if.TGS'; 

1 0, otherwise . 

It is well-known (see e.g. Theorem 7.3 in 25 ) that the bottleneck ratio at set S equals the total 
variation distance between ns and vrgP, i.e Utt^P ~ tt^II = B{S). Hence, the following lemma about the 
metastability of Trg holds. 

Lemma 3.6 Let P be a Markov chain with finite state space fl and let S (^ fl be a subset of states. 
Then, tts is {B(S),l)-meta.stable. 

3.4 Pseudo Mixing Time Tools 

In order to upper bound the mixing time of an ergodic chain, it is often used the fact that, for every 
starting state x € n the total variation distance between the distribution of the chain at time t and the 
stationary distribution n is upper bounded by the maximum, over all states y G fi, of the total variation 
between the chain starting at x and the chain starting at y (see Lemma 4.11 in I25j). i.e. 

\\P\x, •) - 7r||^y < max \\P\x, ■) - P*(y, ■)]] 

In the following lemma we formalize and prove an analogous statement for metastable distributions. 

Lemma 3.7 Let P be a Markov chain with finite state space 51 and let fi be an {e,T) -metastable distri- 
bution supported over a subset S C ft of the state space. Then for every x ^ S and every 1 ^ t ^ T, it 
holds that 

\\P\^^ •) - mIItv ^ £ + max ||F*(x, •) - P\y, -^ . 
II J. V yes -^ ^ 

Proof. From triangle inequality we have 

\\p'{x, •) - 4^^ < \\p\x, •) - /iF*||^y + WfiP' - 4^^ 



Since /x is (e, t)-nietastable for every i ^ T, we have H/xP* — mIItv ^ ^- Observe that, since /i(y) = for 
y ^ S, then for every set of states A C il and for every t it holds that 



\P'{x,A)-i^P\A)\ 



p\x,A)-J2^^iy)P'iy,A) 



yes 



Y,^,{y){P\x,A)-P\y,A)) 



yes 



^ V/i(2/) \P\x,A)~P\y,A)\ ^ max\P\x,A) - P\y,A)\ 
^ yes 



Thus, the total variation between P*(a;, •) and jiP*^ is 



\P\x,■)^^^P%y - —1^*^- "^ ■■^* 



max IP* (a;, A) - nP\A)\ 



< maxniax \P\x,A) - P*-{y,A)\ = max ||P*(x, •) - P*(y, •) 



Aco ye5 



yes 



I TV 



D 
In some cases metastable distributions are concentrated in one single state. The following lemma shows 
that, in those cases, the hitting time of such a state can be used to establish the pseudo-mixing time of 
the metastable distribution. 

Lemma 3.8 Let P be a Markov chain with finite state space Q and let ii be an (e, T)-metastable distri- 
bution concentrated on a single state y. Let Ty he the hitting time of this state. Then for all x ^ ^ and 
1 ^ t ^ T , we have 

\\P\x, •) - ^ll^y ^s+il- e)P, {Ty > t) . 

Proof Since /i is concentrated in y, wc have that 

||P*(x,-)-m||tv = Px{Xt^y) = PAXt^y,Ty^t) + P,iXt^y,Ty>t) 

= P,iXt^y\Ty^ t) P, {Ty S^ f ) + P, {Ty > t) 



Moreover, observe that 

P^,{Xt^y\Tyi^t) ^ Y^ P, {Xt^y\Ty^ k) P, {Ty^k\Tyi^ t) 

= J2PyiXt-k^y)PAry = k\Ty^t) 

k^t 
^ g y^ Pa; (t^ = fc I Ty < t) = £ 



k<t 



where in the inequality wc used the mctastability of fi. Hence, 

||P*(a;,-)-M||Tv = Px{Xt^y\Ty^t)P,{Ty^t) + P,{Ty>t) 

^ eP^ {Ty i^t)+P^{Ty>t)=e+{l- S)P^ {Ty > t) 



D 



4 Ising model on the complete graph 

Consider the following game, that here we call Lsing game: each one of n players has two strategies. 






i ^j 



— 1 and +1, and the utility of player i at profile x = {xi, . . . ,a;„) G { — 1, +1}" is Ui(x) — Xi X^ia^ 
This is the "game-theoretic" formulation of the well-studied Curie- Weiss model (the Ising model on the 
complete graph). 



Observe that for every player i it holds that Mi(x_i, +1) — Ui(x_i, —1) — H{x^i, +1) — H{x^i, —1) 
where -ff(x) = X)/ /c>ei'i"h-''j^fe' hence the Ising game is a potential game with potential function H 

and, as observed in Section [2] the logit dynamics has stationary distribution 7r(x) — e^^^^' /Z. 

The magnetization of x is defined as ^(x) — X]i=i^j' ^^"^ observe that the potential of a profile x 
depends only on its magnetization, i.e. if S'(x) = k then H{x.) — H{k) — i {k"^ — n) . To see this, let us 
name p and m the number of +1 and —1 respectively, in profile x, and observe that p — m = S'(x) = k 
and p + m = n. Each pair of players with the same sign contributes for +1 in i?(x) and each pair of 
players with opposite signs contributes for —1; since there are (2) pairs where both players play +1, (™) 
pairs where both play —1 and p ■ m pairs where players play opposite strategies, we have that 

p\ fni\ 1^^ ^2 



^^^^ "12/ ^12/"^'™" 2*'*'^~"''^ ^ (p + H) 

In this section we study the metastability properties of the logit dynamics for the Ising game (equiv- 
alently the Glauber dynamics for the Curie- Weiss model) from our quantitative point of view. Namely, 
we show that, if we start from a profile where the number of +1 (respectively —1) is a sufficiently large 
majority, and if /3 is large enough then, after an initial pseudo-mixing phase, the distribution of the chain 
at time t is close, in total variation distance, to the degenerate distribution concentrated in the profile 
with all +l's (respectively all — I's) for all t ~ (!?(poly(n)). 

Let 7r+ and 7r_ be the two degenerate distributions concentrated in the states with all +1 and all 
— 1, respectively. The next lemma shows that, for /3 — 57(logn/n), 7r+ and 7r_ are metastable for a 
polynomially-long time. 

Lemma 4.1 If (3 > c\ogn/n then 7r_|_ and 7r_ are (l/n,n''~'^) -metastable. 

Proof. We prove the result for 7r_|_, exactly the same proof (by swapping minuses and pluses) works for 
7r_. Since 7r+(x) = for all x 7^ +1 and 

{0 if S'(x) < {n — 2)/n (i.e if x contains more than one " — 1") 

n ■ i+e/("-3) ^^ "^ (■''■) = (" ^ 2)/n (i.e. if x contains exactly one " — 1") 

l+e4(.-2, if5(x)=l (i.e. ifx=+l) 

the total variation distance between tt+P and 7r+ is 

xe{-i,+i}" 

In the last inequality we used /3 > clogn/n. Hence 7r_|_ is (n^(^~^\ l)-metastable. The thesis follows 
from Lemma 13.21 D 

In order to give an upper bound on the pseudo-mixing time we need some preliminary results about 
birth-and-death chains, which we show in the next subsection. 

4.1 Biased birth-and-death chains 

In this section we consider birth-and-death chains with state space fl = {0, 1, . . . , n} (see Chapter 2.5 
in |25] for a detailed description of such chains). For k G {1, . . . , n — 1} let pfc = Pfc (Xi — k -\- 1), 
q_k = Pfc {Xi = fc — 1), and r^ — 1 — pk — qu = Pfc (-''^i = k). We will be interested in the probability 
that the chain starting at some state h G il hits state before state n, namely Pfe {Xr„ „ = n) where 
To,„ = min{t e N : Xf e {0,n}}. 

We start by giving an exact formula for such probability for the case when pk and qk do not depend 
on fc. 

Lemma 4.2 Suppose for all k S {1, . . . , n — 1} it holds that pk = e and qk = S, for some e and S with 
£ + 5^1. Then the probability the chain hits state n before state starting from state h d fl is 

P,.(X„..=„) = i^. 
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Proof. Let ckfc be the probability to reach state n before state starting from state k, i.e. 

ak = Pfe (^To,„ = n) . 
Observe that for fc = 1 , . . . , n — 1 we have 

ak^ 5 ■ ak-i + s ■ ttfc+i + {1- {S + e)) Uk . (5) 

Hence 

e ■ ak - S ■ ak-i = e ■ ak+i - 5 ■ ak 

with boundary conditions a^ — and «„ = 1. If we name A^ = e ■ a^ — 5 ■ a^-i we have A^ = A^+i for 
all k. By simple calculation and using that ao = it follows that 



i=0 



From an = 1 we get 



Hence 



e-5 



(1 - {5/e)'^) 

1 - {5/ef 
1 - [S/ef 



1 - {5/ef ,_, 

"*; = "i TYTTv^ ■ (o) 

D 



Lemma 4.3 Suppose for all k e {1, . . . , n — 1} it holds that pk ^ s and qk ^ S, for some e and S with 
£ + (5^1. Then the probability to hit state n before state starting from state h G il is 

Proof. Let {Yt} be a birth-and-death chain with the same state space as {Xt} but different transition 
rates 

Pk{Yi^k-l)^5 Pk{Yi=k + l)^e 

Consider the following coupling of Xt and Yt: When {Xt,Yt) is at state {k,h), consider the two [0, 1] 
intervals, each one partitioned in three subintervals as in Fig. 14.11 Let f/ be a uniform r.v. over the 
interval [0, 1] and chose the update for the two chains according to position of U in the two intervals. 

9fe Pk 

I 1 h^ 1 



H 1 



Figure 1: Coupling 

Observe that, since Pfc ^ £ and qk ^ S, if the two chains start at the same state h € Q, i.e. 
{Xq, Yq) = (h, h), then at every time t it holds that Xt ^ Yt. Hence if chain Yt hits state n before state 
0, then chain Xt hits state n before state as well. More formally, let to,„ and fo,„ be the random 
variables indicating the first time chains Xt and Yt respectively hit state or n, hence 

{>fo.„ =n} ^ {Xro_„ - n} 
Thus 

P, (X.,,„ = n) ^ P, (y,„,„ = n) ^ }— [^ ■ 

In the last inequality we used Lemma [4.21 D 
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Lemma 4.4 Suppose for all k G {1, . . . , n — 1} it holds that qk/Pk ^ ol, for some a < 1. Then the 
probability to hit state before state n starting from state h Cz Q is 

Ph (^ro.„ - 0) sC a^ 

Idea of the proof. Let pk — — ^r — and qk — — ^r — and let {Yt} be the birth-and-death chain 

with transition rates pk and qk- Observe that P^ I X^x — O) ~ Pk [ Y^y = ) , indeed it is 

easy to define a coupling of the two chains where chain Xt follows the path traced by chain Yt: 
Let {Ut} be i.i.d. random variables uniformly distributed over [0, 1] and use such variables to 
run chain Yt. As for chain Xt, when at state h toss a coin that gives head with probability 
Ph + Qh] if it gives tail than don't move, if it gives head than move to one of the two adjacent 
states by using the first r.v. Ui that has not been previously used in process Xt. In this way 
it is easy to see that, if they start at the same place, the sequence of states visited by the two 
chains is the same and in the same order. Hence chain Xt hits state before state n if and 
only if chain Yt hits state before state n. 

Finally observe that qk/pk — qk/Pk ^ ct, and since p^ + g^ = 1 it implies that pk ^ 1/(1 + a) 
and qk ^ ct/{l + a). The thesis then follows from Lemma 331 D 

4.2 Convergence time at low temperature 

If Xt is the logit dynamics for the Ising game, the magnetization process St := S{Xt) is itself a Markov 
chain, with state space ft = {—n, —n + 2, • • • , n — 4, n — 2, n}. When at state fc G Jl, the probability to 
go right (to state fc + 2) or left (to state fc — 2) is respectively 

P.(5. = fc + 2)=p. = !^^^-L,^; P.(^i-^-2)=.. = ^^^^^. (7) 

Indeed, let us evaluate the probability to jump from a profile x with magnetization fc to a profile with 
magnetization k + 2. If 5(x) = fc then there are (n + fc)/2 players playing +1 and {n — fc)/2 players 
playing —1. The chain moves to a profile with magnetization fc + 2 if a player playing —1 is selected, this 
happens with probability (n — k)/2n, and she updates her strategy to +1, this happens with probability 



e 



/3«i(x_., + l) I 



g0«,(x_i, + l) _|_ g;3ui(x_,,-l) I _|_ g/3[ni(x_.,-l)-M,(x_,, + l)] 

Finally observe that Ui(x_i, —1) — Mi(x_,:, +1) = — 2^ /■ Xj — —2 (S'(x) — Xi) = — 2(fc + 1). 

For a,b € [—n,n], with a < b, let Ta.b be the random variable indicating the first time the chain 
reaches a state x with x ^ a or x ^ b, 

Ta,b = inf {i eN : Yt !^ a or Yt^b} 

At time Ta,b, chain 1^^ ^ can be in one out of two states, namely the larger state smaller than a or the 
smallest state larger than b. We need to give an upper bound on the probability that when the chain 
exits from interval (a, 6), it happens on the left side of the interval. 

In the next lemma we show that, if the chain starts from a sufficiently large positive state fc, and if 
/3fc^ ^ clogn for a suitable constant c, then when chain Yt gets out of interval (0,n/2), it happens on 
the n/2 side w.h.p. 

Lemma 4.5 Let k ^ Q, be the starting state with 4 ^ fc ^ n/2. If (3 '^ 6/n and l3k'^ ^ 161ogn, then 

Pfe {yn,,.,. ^ 0) sc: 1/n 

Proof. According to ([7]) , the ratio of qh and ph is 

qh _ n + h l + e-2(''+i)/3 
Ph~ n-h' 1 + e^C*-!)/? 



12 



Now observe that for all /i ^ 2 it holds that 

1 + e-^Ci+i)/? 

I ^ p2{h-l)l3 

and for all h ^ ri/2 it holds that 



<; e-'^"-''" ^ e-"P (8) 



n + h ^ l + h/n ^ ^g^/^ 
n — h 1 — h/n ^ 

Hence, for every 2 ^ h ^ n/2 we can give the following upper bound 

Ph ^ 

where in the last inequality we used (3 ^ 6/n. 

Thus, for each state h of the chain with k/2 ^ h ^ n/2 we have that the ratio qh/ph is less than 
e-4/3fe_ jf the chain starts at k, by applying Lemma [4.41 it follows that the probability of reaching k/2 

before reaching n/2 is less than I e^^/^fc j ^ where £ is the number of states between k/2 and k, that is 

£ = k/A. Hence, for every 4 ^ fc ^ n/2, if /3fc^ ^ 16 log n, the chain starting at k hits state n/2 before 
state k/2 w.h.p. 

Pfe {yr.r..,. < k/2) ^ eM'^' ^ 1 

The thesis follows by observing that Pfe (Yr„ „^, s^ O) sC P*. (Yr^^^ ^^^ ^ k/2). D 

In the next lemma we show that, if the chain starts from a state k ^ n/2, and if /3 ^ c log n/n for a 
suitable constant c, then when chain Yt reaches one of the endpoints of interval (0, n) it is on the n side 
with probability exponentially close to 1. 

Lemma 4.6 Let k Cz Q be the starting state with n/2 ^ k ^ n — I. If /3 ^ 8 log n/n, then 

Pfc (>^ro,„ ^ 0) ^ (2/n)"/» 

Proof. Observe that for every h ^ n — 1 it holds that ^^^^ ^ 2n, and by using it together with (|S]) we 
have that Qh/Ph ^ 2ne~^^ for every 2 ^ h ^ n ~ 1. Thus, for every k/2 ^ ft, ^ n — 1 it holds that 
qh/Ph ^ 2ne~^^^ ^ 2/n, where in the last inequality we used k > n/2 and /3 > 8 log n/n. Hence, if the 
chain starts at k, by applying Lemma IT4l it follows that the probability of reaching k/2 before reaching 
n is less than (2/n)^, where t = k/A ^ n/8 is the number of states between k/2 and k. Hence, 

Pfe [Yro,^ ^ 0) sc: Pfe (r,,,,,„ s^ k/2) ^ (2/n)"/8 

D 

In the next lemma we show that for every starting state between and n, the expected time the 
chain reaches or n is at most 0{n^). 

Lemma 4.7 For every fc e fi with k ^ it holds that E^ [to.„] ^ n^. 

Idea of the proof. For the unbiased birth-and-death chain, i.e. when pk — qu ^ 1/2 for all 
fc, it is well-known that the expectation is 0{n^). If the chain has a drift toward one of the 
end points, e.g. Pk = P > 1/2, qu = ^ — P for all fc, then such expected time cannot be larger. 
In the magnetization chain we have qk < Pk for all positive fc, but since pk + q^ < 1 we 
have also to consider the time the chain spends not moving. At state fc the chain moves with 
probability {pk + qk)^ hence the expected time the chain stays in a state before moving is at 
most maxfe{l/(pfc + qk)}- Since pk + qk ^ 1/"- for all fc £ {1, . . . , n — 1} it follows that n^ is 
a (rough) upper bound on the expected time for our chain. D 

Now we can state and prove the main theorem of this section. 



13 



Theorem 4.8 Let x be a profile whose magnetization S'(x) has absolute value |S'(x)| = k. If /3 ^ 
clogn/n and k^ > clog 71//? then 

||P*(x,.)-7r+|| = 0(l/n) 

for every n'* ^ t ^ n'^^^ . 

Proof. Consider w.l.o.g. the case of starting state with positive magnetization, S'(x) == k. Let r„ be the 
first time the chain hits state with all +1 and let ro,n be the first time the magnetization of the chain is 
either n or less than or equal to 0, 

TQ,n = mm{t e N : S{Xt) = n or S{Xt) < 0} 

Since {r„ > f , ro,n ^ t} implies that the magnetization chain reaches before reaching n we have 

Px (t„ >t) = Px (r„ > t, To,„ > i) + Px (r„ > t, To,„ < t) 
s^ Px (ro.„ > t) + Px {S{Xr„J ^ 0) 

Where Yt is the birth- and-death chain with state space J7 and transition rates as in ([7]). As for the first 
term of the smn, from Lemma [4.71 it follows that Ex [To.n] /t ^ 1/n for t ^ n^. As for the second term, 
by conditioning on the position of the chain when it gets out of subinterval (0, n/2) we have 

Pk (i;o,„ ^0) = Pk (!;„,„ ^ 1 1;„ „^, < o) p^ (i;„ „^, ^ o) + 

+Pfe (!;„,„ ^ 1 y,„ „,, ^ n/2) Pfc (y.„ „^, ;^ n/2) 

^ Pfc (^ro,„/. ^ 0) + Pfc (^-o,„ «; 1 1;„,„^, ^ n/2) 
From Lemma [4. 5 1 we have that Pfe (Yrg „/, =^ O) ^ 1/n, and observe that 

Pfc {Yro.,. ^ I y,„ „^, ;? n/2) ^ P„/2 (^ro,„ sJ O) sC (2/n"/^) 



a, a 


c,d 


d,c 


6,6 



where the last inequality follows from Lemma l476l Hence for every i ^ n'' it holds that Px (t„ > t) ^ 3/n. 
Since 7r+ is (1/n, n'^^^)-metastable when /3 ^ clogn/n, the thesis follows from Lemma [3.41 D 

5 Graphical Coordination games on rings 

In this section we study coordination games on a ring |15| . Consider n players identified with the integers 
{0, . . . , n — 1}; the two neighbors of player i are {i — 1) mod n (the left neighbor) and {i + 1) mod n (the 
right neighbor). Every player can pick one of two strategies, and 1, and use it in the two instances of 
the following basic coordination game, played with each of her two neighbors: 

_0 1 


1 

We assume that a > d and b > c which implies that players have an advantage in selecting the same 
strategy of their neighbors and that profiles (0,0) and (1,1) are Nash equilibria of the basic coordination 
game. The utility of a player is the sum of the utilities gained in each basic coordination game she plays. 

The set of profiles of the game is J7 = {0, 1}". We define S'd C fi as the set of profiles where exactly d 
players are playing and i? C fj as the set of profiles in which at least two adjacent players are playing 
0. We also set Sd = Ur=rf ^d- Further, we denote by the profile where all players are playing and by 
1 the profile where all players are playing 1. 

We define A := a — d and S := b — c and assume, w.l.o.g., that A ^ 6. It is easy to see that the 
graphical coordination game on a ring is a potential game with potential function ^(x) ~ X]i=o ^ii^)^ 
where 

{0, if x^ ^ x^+x ; 
(5, if a;^ = x^+x = 1 ; 
A, \ixi = Xi+i = 0. 
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In [5] it is showed that the mixing time of the logit dynamics for this game is fl (e^'^'') . If A > 5, 
then is called the risk dominant strategy [191 and, in this case, the techniques of [5] can be generalized 
to obtain an upper bound to the mixing time that is polynomial in n and exponential in A + (5 and l3. If 
instead A = 6 (and thus no risk dominant strategy exists) an almost matching upper bound is given in 
[2]. These results show that the mixing time is polynomial in n for /? = 0(log7i) and greater than any 
polynomial in n, for (3 = uj (log n). 

5.1 Gaines with Risk Dominant Strategies 

In this section wc study the case A > 5 and prove that for /3 = a;(logn), the logit dynamics reaches in 
polynomial time a metastable distribution and remains close to it for super-polynomial time. On the 
other hand, we know that for /3 = 0{logn) the logit dynamics is rapidly mixing (and thus reaches in 
polynomial time the stationary distribution and stays close to it forever). 

Theorem 5.1 For j3 = Loilogn), for every e > and for every x g fi, the dynamics starting from x 
approaches in polynomial time a distribution that is {e,T(n)), for a super-polynomial function T of n. 

Throughout this section, we set S^ — RLi Sd- 

5.1.1 Metastable Distributions 

The next theorem identifies three classes of metastable distributions. We remind the reader that its is 
the stationary distribution (that is the Gibbs measure defined in Eq. [3]) restricted to the set S of profiles 
(see Eq.lH). 

Theorem 5.2 For every e > 0, tti is {e,e ■ e^^) -metastable and ttq is {e,e ■ e'^'^) -metastable. Moreover, 
for /3 = uj{logn) and constant d> 0, ns* is {e^e^'^"'^°^"''^)-metastable. 



Proof. The bottleneck ratio of 1 is 



Ex/i4i)-P(i.x) _ ^_,,^ 



i3(i)^ — ^- ;; - ' ^e 

7r(l) 

Thus by Lemma 13751 we have that tti is (e"^''^, l)-metastable. By applying Lemma [3.21 we obtain that 
TTi is (e,e • e^'^'^)-metastable for every e > 0. 
Similarly, the bottleneck ratio of is 

7r(0) 

and thus, by applying Lemma 13.61 and Lemma l3.21 we have that ttq is (e, e ■ e^ '^)-metastable for every e. 
Finally, the bottleneck ratio of S'J is 



B[S*a) 



Exea5> ^(^) Eyeo\s* -P(x, y) 
Exes* '^W 



d+i jjiaxxpfls* ''■(x) 
maxxes*7r(x) 

j^d+lg[(n-d-l)5+(d-l)A]/3 

^ -^ (max when d O's are adjacent) 

^ ^d+lp-(n-d-l)(A-5)/3 

^ ,^-n+2(d+i) ^p _ ^(logn) and (A — 5) is a positive constant) 

= e-"("'°s"\ (constant d) 

where 95*^ is the set of all profiles in 5'^ with at least a neighbor in 17 \ 5'^. Thus, by applying Lemma 
and Lemma [3.21 we have that tts* is (e, e^'-"'°s"'')-metastable for every e. D 
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5.1.2 Pseudo-Mixing Time 

In this subsection we look at the the pseudo-mixing time of the metastable distributions described in 
Theorem 15. 21 and we show that, for every starting profile, the dynamics rapidly approaches one of them. 
We remind the reader that the interesting case is (3 = a;(logn) as for /3 = O(logn) the mixing time of 
the logit dynamics is polynomial in n. 
Notation: Not to overburden our notation, we will denote distribution ns* by iTd- 

Our proof distinguishes cases depending on the starting profile x of the chain. We start by considering 
X G 5^, for constant d, and show (see Theorem 15. 7p that the pseudo-mixing time of tt^ is polynomial. On 
the other hand, quite trivially, we can say that, starting from 1, distribution tti is reached immediately. 
Finally, in Theorem 15. 9[ we show that if the dynamics starts from one of the remaining profiles, in 
polynomial time it hits either 5^ or 1 with high probability. 

Starting from S^. From the definition of pseudo mixing time and by Lemma |3.7[ for any 5, we can 
bound t^^ (7) in terms of the total variation distance of two copies of the same Markov chain starting in 
different states. Next lemma relates this quantity, for the logit dynamics we are studying, to the hitting 
time To of profile 0. 

Lemma 5.3 For every S* C 17 and for x, y G S* we have 

||P*(x,-)-P*(y,-)||Tv^2-maxP,(ro ^ t) . 

Thus, by the previous lemma and Lemma 13.71 in order to bound i7rd(7), it is sufficient to give an upper 
bound on Px (tq ^ i), for x G S'J. The next lemma bounds the hitting time of when starting from R. 

Lemma 5.4 For /3 ~ uj (logn), for every A > and every x G i?, we have 

(8-A)n2\ A 

Next we show that, when starting from x G S'd, the dynamics hits R in polynomial time. 
Lemma 5.5 For /3 — uj (logn), for every d > and for every profile x G Sd, 

Px {tr ^ n^) ^ ^^ (1 - negl (n)) . 

Finally, we have 

Lemma 5.6 For j3 = uj (logn), for every d > 0, for x G S'J and for every A > 0, 

8n2 \ 1 A 



To > -;- «; 



A y 2d + 1 4 ■ 

Proof. We need to consider only x E S'^\R. By Lemma 15.41 and Lemma 15.51 we have 
To^—j > Pxfro^ — |rfl^nMPx(rfl^n2) 
> Px ' ' 



TQi^^- ^^jPx(T7jS^n2) 

A\ 2d , ,, ,, 2d A 



4/2d+l' °'''2d+l 4' 

D 
We are now ready to prove an upper bound on the pseudo- mixing time of tt^, when starting from a 
profile in S^. 
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Theorem 5.7 For /? = cj (log 71), for constant d > 1 and for every A > 0, 



S*, ^ Sn-' 

tZii)^ — ^ 



where 7 — 2d+i + '^• 



Proof. By Theorein l5.21 we have that iid is (A/2, 8n^/A)-metastable for every A > and sufficiently large 
n. Therefore, for every x S S'J and t ^ 8n^/A, we have 



P*(x,-)-vrd||^y s; f +maxygs* ||-P*(x,-)-J"(y,-)||Tv (by Lemma [33 
< -I + maxzgs* Pz (ro > t) (by Lemma [573)l 

^ 2ti+i + -^ ■ (tiy Lemma 15.41 and Corollary 15. 6|1 



D 



Starting from outside 5^. Observe that when x = 1, metastable distribution tti is trivially reached 
immediately. Thus, it only remains to analyze x ^ S'J U {1}. For this, it is enough to prove that for such 
an X the hitting time of S'^ U {1} is polynomial. 



Lemma 5.8 For /3 ^ uj (logn), for every d > and for every x € Sd, we have 

1 



Px (ri ^ 7i2) ^ 1 (1 - negl (n)) . 



We can now state the following theorem. 

Theorem 5.9 For 13 ~ lo (log 71), for every d > z > and for every profile x G ^2, we have 

Px {rs*uw ^ n') > (^ + ^) (1 - "^g' (")) ■ 
The discussion above concludes the proof of the Theorem 15.11 

Staying arbitrarly close. We observe that, in Theorem 15.71 the distance between the dynamics and 
the metastable distribution cannot be made arbitrarily small. We can achieve this at the cost of slightly 
reducing the set of starting states from which convergence is proved. Specifically, the next theorem shows 
that, for d = w(l) and arbitrarily small 7 > 0, the logit dynamics starting from S^ is within distance 7 
from TTo in a number of steps that is polynomial in n and in I/7. 

Theorem 5.10 For (3 — a;(logn), d = w(l) and every 7 > 0, 

4^(7)^—. 

7 

Proof. Since (3 — a;(logn). Theorem 15.21 implies that ttq is (7/2, 8n^/7)-metastable for every 7 > and 
sufficiently large n. Therefore, for every x g S"^ and t ^ 871^/7, we have 

|lP*(x,-)-7ro||Tv ^ i+Px(ro>t) (by Lemma [SJ]) 

^ 2^jTi + I7 fbv Lemma 15.41 and Corollarv 

^ 7 . (since d — w(l)) 



D 
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5.2 Games Without Risk Dominant Strategies 

In this section we study the case of graphical coordination games without risk dominant strategies (that 
is, A = (5) played on a ring by n players. Next theorem identifies a class of metastable distributions. 

Theorem 5.11 For every e > and for every ^ d ^ n, distribution fid = — ttq + {^ ^ n) ^i *'' 

(e, ee^^^)-metastahle. 



Proof. We notice that 

llMd-P-Mdllxv 



i5]|(/i.P)(x)-/.,(x)Hl^ 



^i^d{y)P{y,^) -/^d(x) 



d Y. P(0,x) + (l-d)^P(l,x) = ^ 



2A/3 



xGS1„_i 



xeOi 



Thus /i is ( jT^TKiT, 1 ) -metastable. The Theorem follows from Lemma |321 D 

The main and quite surprising result in this section is that for every starting profile x G 5^ the 
dynamics starting in x converges in polynomial time to fid, for d = 1, . . . ,n. In order to prove this 
result, we define tq.i = min{To,ri} and prove that this quantity is polynomial in n with very high 
probability; then we show that the dynamics starting at x £ S*;; after tqi steps is distributed as a 
metastable distribution very close to fid- We formalize these arguments in two technical lemmas. 

Lemma 5.12 If (i ~ uj (logn), then for every x e J7, Px {jo,i ^ n^) 5^ 1 — o(l). 

Lemma 5.13 For every d, li. ^ Sd and /3 = aj(logn), the random variable Xt^ ^ given that Xq = x, has 
distribution v^ = {— -\- Ax) ttq + (l — — — Ax) tti, with |Ax| = o(l). 

The pseudo mixing time of distributions fid, for d = 0, 1, . . . , n, is given by the next Theorem. 
Theorem 5.14 If l3 = uj (logn), for every d and every 7 > 

Proof. From Lemma |5.12[ for every x £ 51 we have 



i^" (x, •) - fid 



TV 



max|Px(X„5 eA)-fid{A)\ 

Acn 

0(1) + max IPx (Xn5 e A I To 1 ^ n''' 
Acfi 



t^d{A)\ 



= o(l) + ||Px(X„5 \tos^ 

s^ o(l) + ||Px(X„5 Iroa s^ 



M 



TV 



^x||tv + ll'^x-Z^dllxv 



where the last inequality follows from the triangle inequality of the total variation distance. Moreover, 
from Lemma 15.131 for every x e S'd, we have 



Ikx-Mrfll 



TV 



|P^''-Hx,-)-Md| 



TV 



o(l) 



Finally, from Theorem 15 . 1 II we have that /ix is (o(l),n^)-metastable and thus. 



|Px (X„s I ro,i ^ n') - 



I TV 






n-'-ro,i 



TV 
0(1). 



TV 



D 
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5.3 Proofs from Section 15.1.21 
5.3.1 Proof of Lemma [STSl 



Proof of Lemma \5.3[ Consider the following partial order ^ over f!,: for profiles x, y G fJ, x ^ y iff for 
every O^i^n— 1, we have that Xi 1^ yi- That is, if x ^ y then x can be obtained from y by changing 
O's into I's. We note that, in according to this order, is the unique maximum. 

We next show that for every two profiles x,y G fJ there exists a monotone (w.r.t. (fi, ^)) coupling 
{Xi , Yi ) of two copies of the logit dynamics for the graphical coordination game on the ring for which 
Xq — X and Iq = y. The Lemma then follows from Theorem I A. 1 1 and Lemma lA.21 

The coupling proceeds as follows: first, pick a player i uniformly at random; then, update the 
strategies Xi and yi of player i in the two chains, by setting 



ixi,yi 



(0,0), with probability min{ai{0 \ x),t7i(0 | y)}; 

(1, 1), with probability min{crj(l | x),(Ti(l | y)} ; 

(0, 1), with probability (7^(0 | x) — min{cri(0 | x), (7^(0 | y)} : 

(1,0), with probability (7^(1 | x) — min{o'i(l | x),cri(l | y)} . 



See Equation[Tl for the definition of Ci. Next we observe that if i is chosen for update, then the marginal 
distributions of Xi and yi agree with (7i{- \ x) and (7i{- \ y), respectively. Indeed, for b G {0,1}, the 
probability that Xi = b is 

min{(Ti{b \ yi),(T,{b \ y)} + (Ti(6 | x) -min{a,{b \ x),ai{b \ y)} = (7^(6 | x) , 

and the probability that yi = b is 

min{(7i(6 | x),CTi(5 | y)} + (7j(l - 6 | x) -min{(7j(l -b \ x),cri(l - 6 | y)} 

= min{cri(6 | x),ai{b | y) + (1 - <7i{b \ x)) - (1 -max{f7i(6 | x),CTi(6 | y)}) = (7i(6 | y) . 

The coupling described above is monotone w.r.t. (J7, :<). Indeed, suppose x ^ y and that the player i 
was selected for update. Since x ^ y, the number of neighbors of i playing in x is less or equal than 
in y and thus i7i(0 | x) ^ (7^(0 | y). Thus, the coupling either sets Xi = yi oi Xi = I and yi = 0. In both 
cases, Xi ^Yi. D 

5.3.2 Proof of Lemma [STU 

Lemma 15.41 gives an upper bound on the hitting time, tq, of 0, for a dynamics starting from a profile 
X G i? (profiles in R are those in which at least two adjacent players play 0). For convenience, we rename 
players so that xq = xi — 0. Intuitively, for /3 — w(logn), each of player and 1 changes her strategy 
with very low probability. Moreover, player 2, when selected for update, plays with high probability. 
Similarly, after player 2 has played 0, we have that each of player 0, 1 and 2 changes her strategy with 
very low probability and player 3, when selected for update, plays with high probability. This process 
repeats until every player is playing 0. In the following, we estimate the number of steps sufficient to 
have all players playing strategy with high probability. 

For sake of compactness, we will denote the strategy of player i at time step t by X^. We start with 
a simple observation that lower bounds the probability that a player picks strategy when selected for 
update, given that at least one of their neighbors is playing 0. 

Observation 5.15 For every player i, if i is selected for update at time t, then, for b G {—1, 1} 

1 



P {XI = I Xl+^ = 0) ^ 1 



l + e(A-5)/^ 



We start by evaluating the probability that the dynamics selects players 2, . . . , n — 1 at least once in 
this order before time t. To this aim, we set pi = and, for i = 2, . . . , n — 1, we define pi as the first 
time player i is selected for update after time step pi-i- Thus, at time pi player i is selected for update 
and players 2, . . . , i — 1 have been selected at least once in this order. In particular, p„_i is the first time 
step at which every player i, i ^ 3, has been selected at least once after his left neighbor. Obviously, 
Pi > Pi-i for i = 2, . . . ,n — 1. The next lemma lower bounds the probability that pn-i ^ t. 
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Lemma 5.16 For every x G i? and every t > 0, we have 

n 

Px {Pn-1 SC t) ^ 1 - — . 

Proof. Every player i has probability - of being selected at any given time step. Therefore, E [^2] = 
E [p2 — pi] = n and E [p, — pt-i] = n, for i = 3, . . . , n — 1. Thus, by linearity of expectation. 



E [Pn-l] = ^ E [p, - p,_i] < 



n\ 



i=2 



The lemma follows from the Markov inequality. D 

Suppose now that t ^ Pn-i- The next lemma shows that, for all players i, the probability that X'l ~ 
is high. 



Lemma 5.17 For every starting profile x £ i?, for every player i and for every time step t > 0, we have 

Px (XI - I pn-l ^t)^(l 



1 



1 + e(^-'5)/5 

We prove the lemma first for i ^ 2. Then we deal with players and 1. 

Fix player i ^ 2, time step t and set Sj+i — t. Starting from time step t and going backward to time 
step 0, we identify, the sequence of time steps Si > Si-i > . . . > S2 > such that, for j = i,i — l, . . .,2, Sj 
is the last time player j has been selected before time Sj+i. We remark that, since t ^ Pn-i > Pi we have 
that players 2, . . . ,i are selected at least once in this order and thus all the s* are well defined. Strictly 
speaking, the sequence Si, . . . ,S2 depends on i and t and thus a more precise, and more cumbersome, 
notation would have been Sij^t. Since player i and time step t will be clear from the context, we drop i 
and t. 

In order to lower bound the probability that X^ = for i ^ 2, we first bound it in terms of the the 
probability that player 2 plays at time S2 and then we evaluate this last quantity. The next lemma is 
the first step. 

Lemma 5.18 For every x e i?, every player 2 ^ i ^ n — 1 and every time step t, we have 

Px {XI = I p„_i ^t)^[i- y^Jk^^ Px {xl = I p„_i < t) . 

Proof. For every fixed i, Si is the last time the player i is selected for update before t and thus X^ = X^, . 
Hence, for i = 2 the lemma obviously holds. For i > 2 and j = 2,...,i, we observe that, since 
t ^ Pn-l > Pi, player j has been selected for update at time Sj and Sj is the last time that player j is 
selected for update before time Sj+i and thus X^ = X|. . 
From Observation 15 . 1 5l we have 



Px [Xi^ = I Pn-l ^tj ^ Px [Xi^ = I Xj-' = 0, Pn-l «: tj . Px [X^-^ = I Pn-l ^ t 

and the lemma follows. D 

We now bound the probability that player 2 plays at time step S2. If player 1 has not been selected 
for update before time S2, then X^^ = Xq = 0, and, from Observation 15.151 we have 

Px {Xl = I Pn-l < t) ^ Px {Xl = I Xi^ = 0, Pn-l < t) 



1 + e(^-^)P 



It remains to consider the case when player 1 has been selected for update at least once before time S2- 
For fixed player i and time step t we define a new sequence of time steps tq > ri , . . . > in the following 
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way. We set ro = S2, and, starting from time step S2 and going backward to time step 0, rj, for j > 0, 
is the last time player j mod 2 has been selected before time rj_i. For the last element in the sequence, 
Tfc, it holds that player {k + 1) mod 2 is not selected before time step r/j. 

Since ri is the last time player 1 has been selected for update before rg = S2, we have X^^ — X^^ 
and, by Observation 15.151 

p,, {xl = I p„_i ^ i) ^ Px {xl = I xl = 0, p„_i s; t) ■ Px {xl = I p„_i ^ t) 

5^ (1 - i + ,}a-s)p ) ■ Px {Xk = I P„-i ^ t) . ^^^ 

Finally, we bound Px (-^^^i = I Pn-i ^ t). 

Lemma 5.19 For every starting profile x G i? and for every time step t, for every fixed player i, let 
To, . . . , rfe he defined as above. If k > 0, we have 



Px (X,\ = I p„_i ^ t) ^ (1 - Y^^^)' 



Proof. For sake of compactness, in this proof we denote the parity of integer a with Via) — a mod 2. 
Thus, the definition of sequence rj gives that player 'P(j) has been selected for update at time rj and 

P, (x,^o) = 1 p„_i < t) ^ Px {xr}'^ ^ 1 x,7^+i) ^ o,p„_i ^ i) • Px (x,^(-'+^) = I p„_i ^ 

li j ^ k player Vij + 1) has not been selected for update between time rj+i and time rj and by 
Observation 15.151 



Px (x,7^) = I p„_i < t) ^ Px (x,';^-'') = I X.'^W+i) = 0,p„_i ^ i) • Px (x.^i:'^^) = I p„_i ^ i 



, instead, player V{k + 1) ha 
Xq ^ ^ = 0. By Observation 15 . 1 5l we have 



If j = k, instead, player V{k + 1) has not been selected for update before time r^ and thus Xr^ 



p, (xr}''> = 1 p„_i ^ t ) ^ Px ( ^e^) = 1 x.^j'^+i) = 0, p„_i ^ t 

1 



^ 1 



l_^e(A-5)/3 



D 



Thus, for every player i ^ 2 and for every time step i > 0, we have 

Px {XI = I pn-i ^t) ^ (1 - YTSri^w) ' Px {Xl = I p„_i s^ t) (from Lemma [EUD 

^ (1 - i+e(i-s), ) ' Px (^,\ - I pn-i < t) (from Equation ED 

(\ i—l-\-k 
1 - T+^(K^m) (froi^ Equation [5311) 

^ (1 - i+e(i-^)/i ) > (since i - 1 + fc s^ t) 

where k is the index of the last term in the sequence rg , ri , . . . previously defined. 

This ends the proof of Lemma [5. 171 for player i ^ 2. The cases j = 0, 1 can be proved in similar way. 
Clearly, if player i has never been selected for update before time t, we have that XI = with probability 
1. If player i has been selected at least once we have to distinguish the cases i = and « = 1. If j = 1, 
we define ro = t + 1 and we identify a sequence of time step ri > r2 > . . . > as above: we have that 
X^ = X^_^ and from Lemma [5. 191 follows that 

1 X'^ / 1 ' 

^ 1 



(x; = I p„_i s^ i) ^ (1-- 
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where k is the last index of the sequence ri , r2 , . . . . Finally, the probability that player plays the 
strategy at time t, given that she was selected for update at least once, can be handled similarly to the 
probability that player 2 plays the strategy at time S2 ■ This concludes the proof of Lemma 15.171 

The following lemma gives the probability that the hitting time of the profile is less or equal to t, 
given that pn-i ^ t. 

Lemma 5.20 For every x € i? and every t > 0, we have 



Px (ro < t I p„_i ^ i) ^ 1 - 



l + g(A-5)/J 



Proof. To prove our lemma we will show a bound on the probability that, conditioned on pn-i ^ t, all 
players are playing at time t. 

Let / be the permutation that sort players in order of last selection for update: i.e., /(O) is the last 
player that is selected for update, /(I) is the next to last one, and so on. We have 



'ri-l 



Px (to ^ t I Pn-1 ^t) ^ Px n Xl^'^ = I p„_i ^ t 

n— 1 / n— 1 

^ n p- (^/*'' = 1 p„_i ^ i) 



J=0 

1 



^ 1- 



where the last inequality follows from Lemma 15.171 D 

Now we are ready to prove Lemma 15.41 

Proof of Lemma \5.4\ From Lemma l5.16l and Lemma l5.20[ we have that for every x € J? and every t > 

Px(T0S=;t) > Px(p„-1 ^t)Px(T0S$t|p«-l<t) 



Thus for every A > 0, we have for t = - — t-^ 



Px (To^t) ^ 1 



t ; V l + e('^-'5)^ 



1 



(8-A)ti3 



8-a; 1 I («-A)«^^ 



> 8(l-f)8-A ^ A 
8-A 8 4' 

where the first inequality follows from the fact that, since /3 = a;(logn), for n large enough, we have 

/3 ^ — ^^ ^"Tj J whereas the second inequality follows from the well known approximation 

l-a^e"T^. D 

5.3.3 Proof of Lemma [STSi 

Let Q* be the first time at which all players have been selected at least once. The following lemma 
directly follows from coupon collector argument; we include a proof for completeness. 

Lemma 5.21 For every i > 0, 

Px(r <i)> 1-ne"*/". 
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Proof. The logit dynamics at each time step selects a player for update uniformly and independently of 
the previous selections. Thus the probability that i players are never selected for update in t steps is 
(l-^)*and 



71 — 1 / ■ \ t n—1 



r>t)^y(l--] scrVe-^scrne-*/". 



^-^ V n J ^-^ 

D 
We define players playing in profile x as the zero-players of x and their neighbors as border-players; 
we also define ^(x) ^ d + f as the number of border-players in x. 

Let T* be the first time step at which a border-player is selected for update before one of its neighboring 
zero-players; if this event does not occur then t* = -|-oo. The next lemma bounds the probability that 
T* is finite given that all players have been selected at least once within time t. 

Lemma 5.22 For every x G 5^ \ i? and every t > 

2d 



Px (r* ^ i I r ^ t) ^ 



2d+l 



Proof. Observe that ii 6* ^ t, then r"^ > i is equivalent to say that t* is infinite: thus we will consider 
F^ir" is finite | 6** < t). 

The proof proceeds by induction on d. Let d ^ 1 and denote by i the one zero-player. Notice that 
T* is finite if and only if one of the two neighbors of i is selected for update before i is selected. Since 
we are conditioning on 6* ^ t, all players are selected at least once by time t and thus the probability of 
this event is 



2 _ 2d 



3 2d-\-l- 

Suppose now that the claim holds for d — f and let x G S'rf \ i?. Denote by Tx the set of all the 
zero-players in x and their border-players and let i be the first player in Tx to be selected for update 
(notice that i is well defined since 9* ^ t). Observe that, if i is a border-player, then r* is finite and this 
happens with probability ,, ^!j . If i is a zero-player, we consider the subset Tx C Tx of the remaining 

d — I zero-players and their border-players, r* is finite only if at least one border-player in Tx is selected 
before one of its neighboring zero-players. Notice though that Tx — Ty, for y G ^'^-i \ R such that 

- f^ ^ __ " 
2j- 



j/j = 1 and y_i = x_i. Thus, by inductive hypothesis, Py (r* is finite \ 9* ^ t) ^ T^- Finally, 



Px (r* is finite \9* ^t) = 77^^ + 77^^ — ; • Py (r* is finite \9* i^t) 



5s 



l{x) d 2d -2 



= 1 



l{x.)+d l{x.)+d 2d -I 
d 



(?(x)-Kd)(2d-l) 
I 



2d+I 



where the last inequality follows from I (x) ^ d + I . D 

Now we are ready to prove Lemma 15.51 
Proof of Leinina [57E[ Suppose r* is finite and let i be the border-player selected for update at time r*. 
Then, at time r*, i has at least one neighbor playing and thus i plays with probability 



Px(X;.=0|r*^t)^(l-^^-l3^) 



Moreover, if i plays strategy 0, then at time t* the dynamics hits a profile in R. Thus, for a finite i > 0, 
we have 

Px {tr < t) ^ Px {X^. ^OAt* ^t) 

= Px (x;. = 1 T* sc: t) Px {t* ^ t) 

Px (^;* = 1 T* ^ i) Px (r* ^ 1 1 r < t) Px {9* s^ t) 

1 \ 2d 

1 + ei'^-S)^) 2d+l 



> 



^ (l- \ .J ^^^fl-ne-*/"), 
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where the last inequahty follows from Lemma 15.211 and Lemma 15.221 Finally, the lemma follows since 
/? — a;(logri) and by taking t = n?. D 

5.3.4 Proof of Lemma [STSl 

This proof is very similar to the proof of Lemma 15.51 in particular, we refer to notation defined in 
Section 15.3.31 If all zero-players are selected before both neighboring border-players, we set t* be the 
time step at which the last zero-players is selected, otherwise we set t* be infinity. 



Lemma 5.23 For every y: £ Sd\ R 



Px(T*^t|rs^<)^l. 



Proof. Observe that ii 6* ^ t, then r* > t is equivalent to say that r* is infinite: thus we will consider 
Px(r* is finite | 6** < t). 

The proof proceeds by induction on d. For the base case, we denote by i the only zero-player and t* 
is finite if and only if i is selected for update before her neighbors. Since we are conditioning on 9* ^ t, 
we know that all players have been selected at least once and thus the probability of this event is i . 

Suppose that the claim holds for d — 1 and let x G Sd\ R- Denote by Tx the set of all the zero- 
players in X along with their border-players and let i be the first player in T^ to be selected for update 
(notice that i is well defined since 6* ^ t and thus all players has been selected at least once). Observe 
that, if i is a border-player, then r* is infinite and this happens with probability u^l - . Otherwise, we 
consider the subset Tx C Tx of the remaining d — 1 zero-players and their border-players, r* will be 
finite only if all zero-players in Tx are selected before their border-players. However, Tx = Ty, where 
y G Sd-i \ R is the profile obtained from x by setting yi — 1. By inductive hypothesis the probability 
Py (t* is finite \9* s^t)^ ^. Thus, we have 

Px (r* is finite \ 9* ^ t) = y-- • Py (r* is finite \ 9* i^ t) 

1 1 
:$ - ■ 



3 3'*-! ■ 

D 
Now we prove Lemma 15.81 
Proof of Lemma \57R Notice that if r* is finite, every time a player is selected for update she have both 
neighbors that are playing and thus 



Px {Xr^ = 1 I r* < f) > 1 - — ^^ > 1 



1 \* / t 



where the last inequality follows from the approximations 1 — a ^ e^^ and 1 — a ^ e^~ 
Obviously, if Xr* = 1, then ti ^ r"^. Thus, for a finite f > 0, we have 

Px (ri ^t) ^ Px {Xr* ^IAt* ^t) 

= Px {Xr* = 1 I r* < t) Px (r-^ ^ t) 

^ Px {Xr* =l\T*^t)P^{T*^t I r ^ t) Px {9* ^ t) 






> (1 - ^TTTT I — (1 - ne 



-t/n 



where the last inequality follows from Lemma 15.211 and Lemma 15.231 Finally, the lemma follows since 
/3 = a;(logn) and by taking t — n^. D 

5.4 Proofs from Section 15.21 

We say that a profile x has a zero-block of size / starting at player i ii Xi = Xj+i = . . . = Xi+i-i = and 
Xi-i = Xi+i — 1. Players i and i + h — 1 are the border players of the block. A similar definition is given 
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for one-blocks. Notice that every profile x 7^ 0, 1 has the same number of zero-blocks and one-blocks and 
this number is called the level of x and is denoted by i'(x). We set i(0) — £{1) = 0. 

The following observation gives the level structure of the potential function (note that we are studying 
the case A = (5). 

Observation 5.24 For every profile x, the potential ofx is <&(x) = (n — 2£(x.))A, regardless of the sizes 
of the zero-blocks and one-blocks. 

Moreover, for a profile x, we defines so(x) as the number of zero-blocks of size 1, si(x) as the number of 
one-blocks of size 1 and set s(x) = so(x) -I- si(x). 

5.4.1 Proof of Lemma 15.121 

We would like to study how long it takes for the logit dynamics to reach or 1. Starting from profile 
X at level z ^ 1, the logit dynamics needs to go down i levels to hit a profile at level 0; and to go down 
one level, it is necessary for one monochromatic block (that is, either a zero-block or a one-block) to 
disappear. We next show that we do not have to wait too long for this to happen. 

Our first step bounds the time Ti needed to go from level z -t- 1 to level i. Consider a profile x at level 
i -\- 1 and number arbitrarily the 2(i -\-l) monochromatic blocks of x and denote by fcj(x) the size of the 
j-th monochromatic block. We define Tij in the following way. Suppose that the dynamics reaches level 
i for the first time after t steps and suppose that this happens because the j-th monochromatic block 
disappears. Then we set Tij = t and Tiji = -l-oo for all j' ^ j. Obviously, for any starting profile x we 
have that 



miuTi 

3 



s$ max Ex [nj \ n,j < nj' for all j' ^ j] , 

j 



Ex [r.] = E: 
For sake of compactness of notation we define 

7, / = max max Ex [t,- ,■ I r,- ,- < r, ^i for all 7' ^ 7I , 

'■ l^K2(i+l)x:£(x)=i+l ^^'-Jl ''J '-'^ J -rj\-, 

kj (x)— / 

set "fi — max/ 7^^/ and observe that Ex [t^] ^ 7^. It is also easy to see that 7^^/ is non-decreasing with I. 
Next we bound 7^ in terms of 7^+1. 

Lemma 5.25 For every i ^ 

7i s^ ri^bi 

where bi = n + "^^^l^^^t^^"^ li+i- 

Proof. 

We next bound ji^i by distinguishing cases depending on the size I. For each case, we let x and j be 
the profile and the monochromatic block that attains the maximum 7^^;. 
1 = 1: 

• if the unique player of the j-th monochromatic block is selected for update and she changes her 
strategy then Tij = 1. This happens with probability ^ • I 1 — . \a,i^ 

• if a neighbor v of the unique player of the j'-th monochromatic block is selected for update and she 
changes her strategy then the dynamics reaches a profile y at same level i -\- 1 and the size of the 
j'-th block increases to 2. If v belongs to a monochromatic block of size 1, this has probability (we 
are conditioning on t^ ^ < Tiji for all j' ^ j); otherwise, the probability is at most 1/2 • 2/n — 1/n. 

• if we select for update a player that is not at the borders of a monochromatic block and she changes 
her strategy, then the dynamics reaches a profile y at level i + 2. This has probability "~ iZ,2a/ 
of occurring. 

• in the remaining cases neither the level nor the length of the j-th monochromatic block changes. 
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Hence, by observing that 7^2 ^ 7i,i we have 

1 / 1 \ 1 , , n-4(i + l) + s(x) 1 

n - 2 n - 4(i + 1) + .s(x) - 1 1 , . , 

(1 + Ti,i)- 



n n 1 + e^A^g 

By shuple calculations and using that n — 4(z + 1) + s(x) ^ 0, we obtain 

^ fl 1 \ / n - 4(i + 1) + s(x) 
7M < (^2 + 4e^A^ + 2 J (," ^ ^"' ^ TT^^^^ ^'^' 

Since f ^ + 4^22^^4:2] «% | for ;5 = w(logn), we have 



l^^ ^ l{i..2 + h) . (10) 



1 < / < n - 2i - 1: 



• if a player at the borders of the j'-th monochromatic block is selected for update (there are two of 
these players) and she changes her strategy (this happens with probability 1/2), then the dynamics 
reaches a profile y at same level i + 1 and the length of the j'-th monochromatic block decreases to 
l-l; 

• if a neighbor v of the border players of the j'-th monochromatic block is selected for update and 
she changes her strategy, then the number of monochromatic blocks does not change (and thus we 
are at still at level i + 1) but the j-th monochromatic block increases in size. 

Notice that, in this case, player v does not belong to a monochromatic block of size 1, since we 
are conditioning on the fact that the j'-th monochromatic block is the first to disappear (r^j < 
Tij' Vj' 7^ j). Therefore the two neighbors of v are playing two different strategies and thus v 
adopts any of the two with probability 1/2. Since there are two players adjacent to the border 
players of block j, this case happens with probability at most l/n. 

• if a player v that is not at the borders of a monochromatic block is selected for update and she 
changes her strategy then the two new adjacent monochromatic blocks are created and the level 
increases 1. Notice that there are n — 4(z + l) + s(x*) such player v and each has probability -^ , JaA/^ 
of changing her strategy. 

• in the remaining cases neither the level nor the length of the j-th monochromatic block changes. 
Hence, 

1 , , 1 , , n - 4(?; + 1) + s(x) 1 

l^,l ^ - 1+7U-1 +- 1+7U+1 + ^ — 1 ^ 2A/3 l+T'+l 

n - 2 n- 4(i + 1) + s(x) 1 \ ,-, , 

n n 1 + e^-^P / 

By simple calculations, similar to the ones for the case / = 1, we obtain 

Ji,i < -^hi,i-i + lt,i+i + bi) . 

From the previous inequality and Equation IIOI a simple induction on I shows that, for every 1 ^ Z < 
n — 2i — 1, we have 

7M^7^((' + l)7W + ^^&.). (11) 
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Moreover, from Eguation llll a simple inductive argument shows that, for every /i ^ 1, 



/ + /^ + i'— ■ 2 ' ^ (j + l)(j + 2) 
l + i ^ + 1,, 

< -; ; 7i/+hH /i&j ■ (12) 

l + h+1 ' ' + 2 ^ ^ 

Z = n — 2i — 1: in this case ah blocks other than the j-th have size 1 and thus every time we select one 
of these players, she changes her strategy with probability (we are conditioning on j being the first 
monochromatic block to disappear). This means that that the size of the j'-th monochromatic block 
cannot increase. Reasoning similar to the ones used in the previous cases, we obtain that 

7i,n-2i-l ^ 7i,n-2J-2 + bi . 

By using Eguation llll we have 

{n-2i-2){n-2i + l) + 2{n-2i) n^ 

li,n-2i-l ^ -Z bi ^ — Oi . 

Finally, for every Z ^ 1, by using Equation 1121 with Zi = n — 2i — 1 — Z, we have 

^ Z + 1 (Z + l)(n-2i-l-Z)^ ^ 2. 

7i,i =^ r-7j.«-2i-i H -z Oj s^n bi. 

n — 2i 2 

D 

Corollary 5.26 If (3 = aj(logn), iZien for every i ^ 0, 7^ = 0{n^) . 

Proof. Note that 6i„/2j-i = n and thus, by using Lemma 15.251 71,1/21-1 ^ n^- Moreover, for ^ i < 
[n/2\ — 1, since n — 4(i + 1) + s(x) ^ n, we have 



ttW^^+0 "^ "' (' + E (t^)j 



The corollary follows by observing that, if /3 = w ( -^^^ I , then the summation in Equation [T^ is o(l). D 

The above corollary gives a polynomial bound to the time that the dynamics take to go from a profile 
at level i + 1 to a profile at level i. Lemma [5.121 easilv follows. 
Proof of Lemma V5.12\ Obviously, for every x at level 1 < fc < ri/2. 



Ex [to.i] s; V max Ey [n] <C V 7, = ©(n'') . 

2 — 2 — 

The lemma follows from the Markov inequality. D 



5.4.2 Proof of Lemma [5^^ 

Proof of Lemma \5.13\ For a profile x, we denote by p^ the probability that the logit dynamics starting 
from X at step tq.i is in profile 0; in other words, Px — Px (^o < '''i)- Trivially, po = 1 and pi = 0. 

Clearly, at time tqi the dynamics is either in the state (this happens with probability px) or in 
the state 1 (this happens with probability 1 - p^). Thus, the state of the dynamics at time tqi when 
starting from profile x is distributed according to the probability distribution 

I'x =Px7ro + (1 -Px)7ri. 

We next show that for /3 — uj{\ogn) and x £ Sd, Px = — + ^x, for some Ax = o(l). By the definition of 
Markov chains we know that 

Px = -P(x,x) •px+ X! Pi^'y)'Py 

yeN{x) 

We then partition the neighborhood N{x) of profile x of level i in 5 subsets, A^i (x) , N2 (x) , N3 (x) , N4 (x) , N^ (x) 
such that, for two profiles yi,y2 in the same subsets it holds that P(x, yi) = P(x, y2). 
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• A^i (x) is the set of profiles y obtained from x by changing the strategy of a player of a zero-block 
of size 1. Observe that |A^i(x)| = so(x). Moreover, for every y 6 A^i(x), y is at level i — 1, has 
|x|o - 1 players playing and P(x,y) = i • (1 - jip^W)- 

• N2 (x) is the set of profiles y obtained from x by changing the strategy of a player of a one-block 
of size 1. Observe that |iV2(x)| = si(x). Moreover, for every y G A^2(x), y is at level i — 1, has 
|x|o -I- 1 players playing and P(x, y) == i • (1 - j^^^kw)- 

• N3 (x) is the set of profiles y obtained from x by changing the strategy of a border player of a zero- 
block of size greater than 1. Observe that |iV3(x)| = 2(i — so(x)). Moreover, for every y g 7V3(x), 
y is at level i, and has |x|o — 1 players playing and P(x,y) = l/2n. 

• N4{x.) is the set of profiles y obtained from x by changing the strategy of a border player of a one- 
block of size greater than 1. Observe that |iV4(x)| = 2{i — si(x)). Moreover, for every y G N4{x), 
y is at level i, and has |x|o -I- 1 players playing and P(x,y) = l/2n. 

• N^{x) is the set of all the profiles y G N{x.) that do not belong to any of the previous 4 subsets. 
Observe that |iV5(x)| = n — ii + s(x). Moreover, for every y e A^5(x), y is at level i -|- 1, and 

"(X, y) = - • l_|_e2A/3 • 

Moreover, we have that 

s(x) 1 2i — s(x) n — 4i + s(x) 



P(x,x) 
Then, we have 



l_^g2A/3 ■ n n \ l + e^'^f^ 



n 



^(i-T^W) E Py+ E ^H+^ E Py+ E Py] 

^ ^ \yeNi(^) yeiV2(x) / \yeJV3(x) yGiV4(x) / 

1 1 Y^ 

+ r, 1 + e2A;3 Z^ Py 



yeJV5(x) 




s(x) 1 2i — s(x) n — 4i -f s(x) . 



n 1 + e^^P n n \ 1 + e^^" 

E Py^ E Py\^^\ E Py^ E Py\ 

;iVi(x) yeiV2(x) / \yeiV3(x) yeiV4(x) / 



n — 2i c ,, ,, 

n 1 + e'"^P 



where 





E Py - (" - 40Px 

yeAfi(x)UA'2(x) 

We notice that, since 1 ^ i ^ n/2 and |A'^i(x)| + |A''2(x)|, |iV5(x)| ^ n, we have \c\ ^ 2 and thus the last 
term in Equation [TJ] is negligible in n (since /3 — uj{\ogn)). Hence we have that the following condition 
holds for every level i ^ 1 and every profile x at level i: 

E Py+ E Py 

yyeAfi(x) yeJV2(x) 

where 77x is negligible in n. This gives us a linear system of equations in which the number of equations 
is the same that the number of variables. 

Next we find a solution to a "modified" version of above system, where we omit the negligible part 
in every equation, and then we show that this solution cannot be very different from the solution of the 
"original" system. 

We build the solution for the "modified" system inductively on the level i: for every profile x G iSd at 
level (this is only possible for d = or d = n), we have, as discussed above, Px = -^^ Now, we assume 
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that for every profile x S 5'd at level i — 1, Px — — is a. solution for the system. For x G S'd at level i, we 
can rewrite the "modified" condition as follows: 

Equation [15] gives another linear system of equations. This system has a unique solution: indeed, it 
has the same number of equations and variables and the matrix of coefficients is a diagonally dominant 
matrix (since |iV3(x) U 7V4(x)| ^ Ai) and thus it is nonsingular. Moreover, if we set, for every profile x 
at level i, p^ — ^^, then the right hand side of the Equation [TS] becomes 

so(x)(i— 1 si(x)(i+l i — so(x)d— 1 i — si(x)(i+l d 
2i n 2i n 2i n 2i n n ' 

and hence the system is satisfied by this assignment. Summarizing, we have found that the "modified" 
system has a unique solution p^ = ^^ for every profile x. Now, let p* = px + Ax be the assignment that 
satisfies all "original" conditions: since, as n grows unbounded, these conditions approach the "modified" 
ones, we have that p* ^^^ to approach to p^ and thus we have that |Ax| = o(l) for every profile x. D 

6 The OR game 

The OR-game is a toy n-player game where every player has two strategies, say {0, 1}, and each player 
pays the OR of the strategies of all players (including herself) . More formally, the utility of player i e [n] 
is Ui(0) = and Ui(x) = — 1 for every x 7^ 0. 

In [3] we showed that the mixing time of the logit dynamics for the OR-game is roughly e^ for 
/3 = ©(logn) and it is roughly 2" for larger (3. Here we study the metastability properties of the 
OR game to highlight the distinguishing features of our quantitative notion of metastability based on 
distributions. Namely, we show that if we start the logit dynamics at a profile where at least one player 
is playing 1, then after Oilogn) time steps the distribution of the chain is close to uniform, and it stays 
close to uniform for exponential time. Hence, even if there is no small set of the state space where 
the chain stays close for a long time, we can still say that the chain is "metastable" meaning that the 
"distribution" of the chain stays close to some well-defined distribution for a long time. 

6.1 Ehrenfest urns 

We first need two simple lemmas that will be used in the proof of Theorem l6.4l 

The Ehrenfest urn is the Markov chain with state space i7 = {0, 1, . . . , n} that, when at state k, moves 
to state fc — 1 or fc -|- 1 with probability k/n and {n — k)/n respectively (see, for example. Section 2.3 
in [25] for a detailed description). The next lemma gives an upper bound on the probability that the 
Ehrenfest urn starting at state k hits state within time step t. 

Lemma 6.1 Let {Zt} be the Ehrenfest urn over {0, 1, . . . , n} and let tq be the first time the chain hits 
state 0. Then for every k ^ 1 it holds that 

c' 
Pfc (tq < n log n + en) < — 

n 

for suitable positive constants c and c' . 

Proof. First observe that for any t J^ 3 the probability of hitting before time t for the chain starting 
at 1 is only 0{l/n) larger than for the chain starting at 2, which in turn is only 0{l/n) larger than for 
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the chain starting at 3. Indeed, by conditioning on the first step of the chain, we have 

Pi (to < t) = Pi (to < 1 1 Zi - 0) Pi {Zi = 0) + Pi (to < 1 1 Zi - 2) Pi {Zi - 2) 

= - + "^^——^2 (to < t - 1) ^ - + P2 (to < t) 
n n n 

P2(To<t) = P2(To<t|Zi = l)P2(Zi = l)+P2(To<t|Zi=3)P2(Zi=3) 

= -Pi (to < i - 1) + P3 (to < t - 1) 

n n 

^ - - + P2 To < t + P3 To < t) 

n \n J n 

Hence, 

P2 (to <t) ^ + P3 (to < t) s; - + P3 (to < t) 

n — 2 n 

Pi (to < i) ^ - + P3 (to < t) 

n 

Moreover observe that the probability that the chain starting at k hits state before time t is 
decreasing in k, in particular, for every fc ^ 3 it holds that P^ (to < i) ^ P3 (to < t). Now we show that 
P3 (to < nlogn + en) = 0{l/n) and this will complete the proof. 

Let us consider a path V of length t starting at state 3 and ending at state 0. Observe that any such 
path must contain the sub-path going from state 3 to state whose probability is 6/n^. Moreover, for 
all the other i — 3 moves we have that if the chain crosses an edge {i,i + l) from left to right then it must 
cross the same edge from right to left (and viceversa). The probability for any such pair of moves is 

n — i i + 1 e-*-/" 



for every i. Hence, for any path V of length t going from 3 to 0, the probability that the chain follows 
exactly path V ici 

„ ,,^ ^, ^, 6 /e2/"V*"'^^' 6 23 /ei/«\* 48 /e^'^ 



n"' \ 4 / n"' 



Let £ and r be the number of left and right moves respectively in path V then £ + r = t and £ ~ r = 3. 
Hence the total number of paths of length t going from 3 to is less than 

Thus, the probability that starting from 3 the chain hits for the first time exactly at time t is 



t \ 48 /e^/' 



Finally, the probability that the hitting time of is less than t is 



P3 (to <t) s^ y^ P3 (to = i) 




< 48^^,/„_48 e*/"- 


-1 
- 1 



486^= 



i=3 



In the last inequality we used that e^'" — 1 ^ 1/n and t — nlogn + en. D 

In the proof of Theorem 16.41 we will be dealing with the lazy version of the Ehrenfest urn. The next 
lemma, which is folklore, allows us to use the bound we achieved in Lemma [6. II for the non-lazy chain. 



Notice that such probability is zero if t — 3 is odd 
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Lemma 6.2 Let {Xt} be an irreducible Markov chain with finite state space f2 and transition matrix P 
and let {Xt} be its lazy version, i.e. the Markov chain with the same state space and transition matrix 
P = -^-i^ where I is the Q, xVt identity matrix. Let Ta and fa be the hitting time of state a € ft in chains 
{Xt} and {Xt} respectively. Then, for every starting state 5 G f2 and for every time t eN it holds that 

Pb {fa ^t)i^Pb {Ta S$ t) 

6.2 OR-game metastability 

The next lemma shows that, if we start from the uniform distribution, the distribution of the logit 
dynamics stays e-close to uniform for e2" time steps. 

Lemma 6.3 Let P be the transition matrix of the logit dynamics for the n-player OR-game, let U be 
the uniform distribution over {0, 1}". Then U is {s,e2'")-metastable. 

Proof. Observe that, by starting from the stationary distribution, the probability of being in y G {0, 1}" 
after one step of the chain is 

r 2-" if |y| ^ 2 

UP{y)= Y: C/(x)P(x,y) = i, J^ P(x,y)= 2- (^ + ^T^) if IyI = 1 

x6{o,i}" xe{o,i}" 1^ 2-"(^^) ify = 

Hence, the total variation distance between the uniform distribution and the distribution of the chain 
after one step is 

\\up-u\\ = ^ E |f/^(y)-c/(y)l=2-"^^^2-« 

ye{o,i}" 

Thus, the uniform distribution is (2~", l)-metastable and the thesis follows from Lemma [3.21 D 

In the next theorem we show that, if the chain starts from a state containing at least one 1, then 
after O(log n) time steps the distribution of the chain is e-close to the uniform distribution, and it stays 
£-close to uniform for exponential time. 

Theorem 6.4 Let P be the transition matrix of the logit dynamics for the n-player OR-game, let U be 
the uniform distribution over {0, 1}", let x G {0, 1}" with |x| = fc ^ 1 be the starting state of the chain, 
and let e > 0, then it holds that 

\\P\K,-)-U\\^e 

for every time t such that n\og{3n/e) ^t^ e2"~^. 

Proof. Let {Xt} be the Markov chain starting at x and let {Yt} be a lazy random walk on the n-cube 
starting at the uniform distribution, so that Xt is distributed according to -P*(x, •) and Yt is uniformly 
distributed over {0, 1}". Consider the following coupling {Xt, Yt): when chain {Xt} is at state y G {0, 1}" 
then choose a position i G [n] u.a.r. and 

• If |y| > 2 thciifl, choose an action a G {0, 1} u.a.r. and update both chains Xt and Yt in position i 
with action a; 

• If |y| = 1 then 

— if Xt has in position i than proceed as in the previous case; 

— if Xt has 1 in position i then 

* update both chains at in position i with probability 1/2; 

* update both chains at 1 in position i with probability 1/(1 + e^); 

* update chain Xt at and chain Yt at 1 in position i with probability 
1/(1 + e-'3)- 1/2. 



'With |y| we mean the number of I's in profile y 
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• If |y| = then 

— update both chains at in position i with probabihty 1/2; 

— update both chains at 1 in position i with probabihty 1/(1 + e°); 

— update chain Xt at and chain Yt at 1 in position i with probabihty 
1/(1 + e-'5)- 1/2. 

By construction we have that {Xt,Yt) is acouphngof P*(x, •) and U, hence l|-P*(x, •)— 1/|| < Px,£/ {Xt ^ Yt) 
Moreover observe that, if at time t all players have been selected at least once and chain Xt has not 
yet hit profile = (0, • • • , 0) G {0, 1}", then the two random variables Xt and Yt have the same value. 
Hence 

||P*(x,.)-;7|| «; P^^uiXty^Yt) 

^ Px,[/ (to ^ t U ry < t) 

< Px,C/ (to < t) + Px,C/ iv < t) 

where tq is the hitting time of for chain Xt, and 77 is the first time all players have been selected at 
least once. 

From the coupon collector's argument it follows that for every t ^ nlog(3n/e) 

Px,c/ {V<t)^ e/3 (16) 

As for the second term observe that Px,c/ (tq ^ i) ^ Pfc {po ^ t) where po is the hitting time of state 
for the lazy Ehrenfest urn. More formally, consider the equivalence relation over J7 = {0, 1}" such that 
two profiles x and y are equivalent if they have the same number of I's and let {Zt} be the projection 
of chain {Xt} over the quotient space fl^ — {0, 1, . . . ,n} of such equivalence relation. Then {Zt} is a 
Markov chain with state space ^l^ and transition matrix 

P#(z,z-1) = ^; P#(*,z) = -; P#(*,* + l) = ::;^; for* = 2,...,n (17) 



:-; P#(*,* + l) = 


n — I 


2n ' 


'.(M) = VV . 


1 

-1 , fi\ 



and 

„/N 1 1 „,,n— 1 1 „ , , n ~ 1 

P# 1,0 = ^- P#l,l =^^+ f,_^ ,. P#l,2-^— 

n{l + e P) n 2n n(l-\-eP) 2n 

The hitting time tq of state £ J7 for chain {Xt} coincide with the hitting time po of state G 51^ for 
the projection Zt- 

Observe that, from the transition probabilities in (|17p. chain {Zt} is almost the lazy Ehrenfest urn, 
the only difference being at states 1 and 0. Moreover, the transition from state 1 to state in the Zt 
holds with probability smaller than the probability of the same transition in the Ehrenfest urn. From 
Lemmas 16.11 and 16.21 it follows that 

Px,;7 (tq < n log n + 71 log(3/e)) ^ c/n (18) 

for a suitable constant c = c(e). Hence, for t = nXogn + nlog(3/e), by combining ([T5| and ([T5| it holds 
that 

||p*(x,.)-f/|K| + -^| 

6 n 2 
for n sufficiently large. 

Since from Lemma [6.31 we have that the uniform distribution is (£/2,£2"~-'^)-metastable, the thesis 
follows by applying Lemma 13.41 D 

7 Conclusions and open problems 

Logit dynamics is a clean and tractable model of evolution for strategic games where players have limited 
rationality. The stationary distribution of the induced Markov chain is the natural long-term equilibrium 
concept for games under logit dynamics. However, when the mixing time is long, the behavior of the 
Markov chain in the transient phase becomes important and it is worth looking for "regularities" at a 
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time-scale shorter than mixing time. Such regularities have been previously explored, for some classes 
of Markov chains, by means of "metastable states" . We believe that a more general and useful concept 
is that of "metastable distributions" . 

In this paper we defined a quantitative notion of metastable distribution and we analyzed the metasta- 
bility properties of the logit dynamics for some classes of coordination games. We showed that, even 
when the mixing time is exponential, it is possible to find some distributions that well-approximate the 
distribution of the chain for a time-window of polynomial size. Such metastable distributions can be 
found even in the case of the OR-game, where no partition of the state space in metastable states exists. 
A natural open question is whether the metastability properties for coordination games we observed in 
this paper hold in general for potential games. 

In the case of the Ising model on the complete graph, we showed that when /3 > clogn/n the 
two degenerate distributions are metastable for poly{n) time and they are quickly reached from a large 
fraction of the state space. It would be interesting to investigate the metastability properties when 1/n < 
/3 < log n/n. Indeed, in that range the mixing time is exponential but the distributions concentrated in 
the two extremal states are not metastable. 
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Appendix 



A Markov chain summary 

In this section we recall some basic facts about Markov chains. Fore a more detailed description and for 
notation conventions we refer the reader to [25]. 

Consider a Markov chain A4 with finite state space Q and transition matrix P. It is a classical 
result that if Ai is irreducible and aperiodiq^ there exists an unique stationary distribution; that is, a 
distribution tt on 51 such that tt • P = tt. 
The total variation distance IJA* ~ ^IItv between two probability distributions /x and i/ on fi is defined as 

An irreducible and aperiodic Markov chain M converges to its stationary distribution tt; specifically, 
there exists < q < 1 such that 

d{t) < a*, 

where 

d(t) = max ||P*(x, •) — ttII 

and P'^ix, •) is the distribution at time t of the Markov chain starting at x. For < e < 1/2, the mixing 
time is defined as 

tmix(e) = min{f e N : d{t) < e}. 

It is usual to set s = 1/4 or e = l/2e. If not explicitly specified, when we write imix we mean tmix(l/4). 
Observe that imix(£) < [logj e^^limix- 
Coupling. A coupling of two probability distributions fi and ly on Q is a, pair of random variables 
(X, Y) defined on 51 x il such that the marginal distribution of X is /it and the marginal distribution 
of Y is ly. A coupling of a Markov chain M. with transition matrix P is a process {Xt, Yt)'^^ with the 
property that both Xt and Yt are Markov chains with transition matrix P. When the two coupled chains 
start at (A'o,io) = ix,y), we write Px.y (■) and 'Ei^^y [■] for the probability and the expectation on the 
space where the two chains are both defined. 
We denote by Tcoupic the first time the two chains meet; that is, 

Tcoupic = min{t : Xt^Yt} . 

We will consider only couplings of Markov chains with the property that for s ^ Tcoupic, it holds X^ — Y^. 
The following theorem establish the importance of this tool (see, for example. Theorem 5.2 in [25 ). 

Theorem A.l (Coupling) Let A4 be a Markov chain with finite state space 51 and transition matrix 
P. For each pair of states x,y G 51 consider a coupling {Xt,Yt) of M with starting states Xq = x and 
Yq — y. Then 

\\P\x, ■) - P\y, OIItv ^ P^:y (Tcouple > t) . 

Consider a partial order ^ over the states in 51. A coupling of a Markov chain is said to be monotone 
w.r.t. (51, <) if, for every t ^ 0, Xt ^ Yt ^ Xt+i r< Yt+i. For a state 2 g 51, the hitting time t^ of z is 
the first time the chain is in state z, t^ — inf{i : Xt — z}. The following theorem holds. 

Lemma A. 2 Let M be a Markov chain with finite state space 51 and transition matrix P. Let ^ be a 
partial order over 51. For each pair of states x,y ^ U, consider a coupling [Xt^Yt) of Ai with starting 
states Xq — x and Yq — y that is monotone w.r.t. (51, :<). Moreover, suppose the ordered set (fi, ^) has 
an unique maximum at z. Then 

Px^y {Tcouple > i) < 2 • maxjP^; (r2 > t) ,Py (t^ > t)} . 



* Roughly speaking, a finite-state Markov chain is irreducible and aperiodic if there is a time t such that, for all pairs of 
states X, y, the probability to be in y after t steps, starting from x, is positive. 
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